Method and apparatus for improving the coding of side information required for coding a higher order ambisonics representation of a sound field

ABSTRACT

Higher Order Ambisonics represents three-dimensional sound independent of a specific loudspeaker set-up. However, transmission of an HOA representation results in a very high bit rate. Therefore compression with a fixed number of channels is used, in which directional and ambient signal components are processed differently. For coding, portions of the original HOA representation are predicted from the directional signal components. This prediction provides side information which is required for a corresponding decoding. By using some additional specific purpose bits, a known side information coding processing is improved in that the required number of bits for coding that side information is reduced on average.

TECHNICAL FIELD

The invention relates to a method and to an apparatus for improving thecoding of side information required for coding a Higher Order Ambisonicsrepresentation of a sound field.

BACKGROUND

Higher Order Ambisonics (HOA) offers one possibility to representthree-dimensional sound among other techniques like wave field synthesis(WFS) or channel based approaches like the 22.2 multichannel audioformat. In contrast to channel based methods, the HOA representationoffers the advantage of being independent of a specific loudspeakerset-up. This flexibility, however, is at the expense of a decodingprocess which is required for the playback of the HOA representation ona particular loudspeaker set-up. Compared to the WFS approach, where thenumber of required loudspeakers is usually very large, HOA signals mayalso be rendered to set-ups consisting of only few loudspeakers. Afurther advantage of HOA is that the same representation can also beemployed without any modification for binaural rendering to head-phones.

HOA is based on the representation of the spatial density of complexharmonic plane wave amplitudes by a truncated Spherical Harmonics (SH)expansion. Each expansion coefficient is a function of angularfrequency, which can be equivalently represented by a time domainfunction. Hence, without loss of generality, the complete HOA soundfield representation actually can be assumed to consist of O time domainfunctions, where O denotes the number of expansion coefficients. Thesetime domain functions will be equivalently referred to as HOAcoefficient sequences or as HOA channels in the following.

The spatial resolution of the HOA representation improves with a growingmaximum order N of the expansion. Unfortunately, the number of expansioncoefficients O grows quadratically with the order N, in particularO=(N±1)². For example, typical HOA representations using order N=4require O=25 HOA (expansion) coefficients. According to the previouslymade considerations, the total bit rate for the transmission of HOArepresentation, given a desired single-channel sampling rate f_(S) andthe number of bits N_(b) per sample, is determined by O·f_(S)·N_(b).Consequently, transmitting an HOA representation of order N=4 with asampling rate of f_(S)=48 kHz employing N_(b)=16 bits per sample resultsin a bit rate of 19.2 MBits/s, which is very high for many practicalapplications like e.g. streaming. Thus, compression of HOArepresentations is highly desirable.

The compression of HOA sound field representations is proposed in WO2013/171083 A1, EP 13305558.2 and PCT/EP2013/075559. These processingshave in common that they perform a sound field analysis and decomposethe given HOA representation into a directional component and a residualambient component. On one hand the final compressed representation isassumed to consist of a number of quantised signals, resulting from theperceptual coding of the directional signals and relevant coefficientsequences of the ambient HOA component. On the other hand it is assumedto comprise additional side information related to the quantisedsignals, which side information is necessary for the reconstruction ofthe HOA representation from its compressed version.

An important part of that side information is a description of aprediction of portions of the original HOA representation from thedirectional signals. Since for this prediction the original HOArepresentation is assumed to be equivalently represented by a number ofspatially dispersed general plane waves impinging from spatiallyuniformly distributed directions, the prediction is referred to asspatial prediction in the following.

The coding of such side information related to spatial prediction isdescribed in ISO/IEC JTC1/SC29/WG11, N14061, “Working Draft Text ofMPEG-H 3D Audio HOA RM0”, November 2013, Geneva, Switzerland. However,this state-of-the-art coding of the side information is ratherinefficient.

SUMMARY OF INVENTION

A problem to be solved by the invention is to provide a more efficientway of coding side information related to that spatial prediction.

This problem is solved by the methods disclosed in claims 1 and 6. Anapparatus that utilises these methods is disclosed in claims 2 and 7.

A bit is prepended to the coded side information representation dataζ_(COD), which bit signals whether or not any prediction is to beperformed. This feature reduces over time the average bit rate for thetransmission of the ζ_(COD) data. Further, in specific situations,instead of using a bit array indicating for each direction if theprediction is performed or not, it is more efficient to transmit ortransfer the number of active predictions and the respective indices. Asingle bit can be used for indicating in which way the indices ofdirections are coded for which a prediction is supposed to be performed.On average, this operation over time further reduces the bit rate forthe transmission of the ζ_(COD) data.

In principle, the inventive method is suited for improving the coding ofside information required for coding a Higher Order Ambisonicsrepresentation of a sound field, denoted HOA, with input time frames ofHOA coefficient sequences, wherein dominant directional signals as wellas a residual ambient HOA component are determined and a prediction isused for said dominant directional signals, thereby providing, for acoded frame of HOA coefficients, side information data describing saidprediction, and wherein said side information data can include:

-   -   a bit array indicating whether or not for a direction a        prediction is performed;    -   a bit array in which each bit indicates, for the directions        where a prediction is to be performed, the kind of the        prediction;    -   a data array whose elements denote, for the predictions to be        performed, indices of the directional signals to be used;    -   a data array whose elements represent quantised scaling factors,

said method including the step:

-   -   providing a bit value indicating whether or not said prediction        is to be performed;    -   if no prediction is to be performed, omitting said bit arrays        and said data arrays in said side information data;    -   if said prediction is to be performed, providing a bit value        indicating whether or not, instead of said bit array indicating        whether or not for a direction a prediction is performed, a        number of active predictions and a data array containing the        indices of directions where a prediction is to be performed are        included in said side information data.

In principle the inventive apparatus is suited for improving the codingof side information required for coding a Higher Order Ambisonicsrepresentation of a sound field, denoted HOA, with input time frames ofHOA coefficient sequences, wherein dominant directional signals as wellas a residual ambient HOA component are determined and a prediction isused for said dominant directional signals, thereby providing, for acoded frame of HOA coefficients, side information data describing saidprediction, and wherein said side information data can include:

-   -   a bit array indicating whether or not for a direction a        prediction is performed;    -   a bit array in which each bit indicates, for the directions        where a prediction is to be performed, the kind of the        prediction;    -   a data array whose elements denote, for the predictions to be        performed, indices of the directional signals to be used;    -   a data array whose elements represent quantised scaling factors,

said apparatus including means which:

-   -   provide a bit value indicating whether or not said prediction is        to be performed;    -   if no prediction is to be performed, omit said bit arrays and        said data arrays in said side information data;    -   if said prediction is to be performed, provide a bit value        indicating whether or not, instead of said bit array indicating        whether or not for a direction a prediction is performed, a        number of active predictions and a data array containing the        indices of directions where a prediction is to be performed are        included in said side information data.

An aspect of the invention relates to a method for decoding a bitstreamincluding encoded HOA representations. The method includes evaluating avalue of a bit KindOfCodedPredIds; evaluating, based on the value of thebit KindOfCodedPredIds, a first array ActivePred, wherein each elementof the first array ActivePred indicates if, for a correspondingdirection, a prediction is performed; determining, based on theevaluation of the first array ActivePred, elements of a vector p_(type);evaluating a second array PredDirSigIds, wherein elements of the secondarray PredDirSigIds denote indices of directional signals to be used foractive predictions; determining, based on the vector p_(type) and theelements of the second array PredDirSigIds, elements of a matrix PINEdenoting indices from which directional signals a prediction for adirection is to be performed. An aspect of the invention may furtherrelate to apparatus and/or non-transitory computer readable medium codeconfigured to perform this method.

Each element of the second array PredDirSigIds may denote, for thepredictions to be performed, indices of the directional signals to beused and wherein each element was coded based on ┌log₂(|{tilde over(D)}_(ACT)+1|)┐ bits, and is correspondingly decoded, wherein {tildeover (D)}_(ACT) denotes a number of elements of said data set of indicesof directional signals.

Advantageous additional embodiments of the invention are disclosed inthe respective dependent claims.

BRIEF DESCRIPTION OF DRAWINGS

Exemplary embodiments of the invention are described with reference tothe following accompanying drawings:

FIG. 1 illustrates an exemplary coding of side information related tospatial prediction in the HOA compression processing described in EP13305558.2;

FIG. 2 illustrates an exemplary decoding of side information related tospatial prediction in the HOA decompression processing described inpatent application EP 13305558.2;

FIG. 3 illustrates an HOA decomposition as described in patentapplication PCT/EP2013/075559;

FIG. 4 depicts an illustration of directions (depicted as crosses) ofgeneral plane waves representing the residual signal and the directions(depicted as circles) of dominant sound sources. The directions arepresented in a three-dimensional coordinate system as sampling positionson the unit sphere;

FIG. 5 illustrates a state of art coding of spatial prediction sideinformation;

FIG. 6 illustrates an inventive coding of spatial prediction sideinformation;

FIG. 7 illustrates inventive decoding of coded spatial prediction sideinformation;

FIG. 8 is continuation of FIG. 7.

DESCRIPTION OF EMBODIMENTS

In the following, the HOA compression and decompression processingdescribed in patent application EP 13305558.2 is recapitulated in orderto provide the context in which the inventive coding of side informationrelated to spatial prediction is used.

HOA Compression

In FIG. 1 it is illustrated how the coding of side information relatedto spatial prediction can be embedded into the HOA compressionprocessing described patent application EP 13305558.2.

For the HOA representation compression, a frame-wise processing withnon-overlapping input frames C(k) of HOA coefficient sequences of lengthL is assumed, where k denotes the frame index. The first step or stage11/12 in FIG. 1 is optional and consists of concatenating thenon-overlapping k-th and (k−1)-th frames of HOA coefficient sequencesC(k) into a long frame {tilde over (C)}(k) as

{tilde over (C)}(k):=[C(k−1)C(k)],  (1)

which long frame is 50% overlapped with an adjacent long frame and whichlong frame is successively used for the estimation of dominant soundsource directions. Similar to the notation for {tilde over (C)}(k), thetilde symbol is used in the following description for indicating thatthe respective quantity refers to long overlapping frames. If step/stage11/12 is not present, the tilde symbol has no specific meaning.

A parameter in bold means a set of values, e.g. a matrix or a vector.

The long frame {tilde over (C)}(k) is successively used in step or stage13 for the estimation of dominant sound source directions as describedin EP 13305558.2. This estimation provides a data set

_(DIR,ACT)(k)⊆{1, . . . , D} of indices of the related directionalsignals that have been detected, as well as a data set

_(Ω,ACT)(k) of the corresponding direction estimates of the directionalsignals. D denotes the maximum number of directional signals that has tobe set before starting the HOA compression and that can be handled inthe known processing which follows.

In step or stage 14, the current (long) frame {tilde over (C)}(k) of HOAcoefficient sequences is decomposed (as proposed in EP 13305156.5) intoa number of directional signals X_(DIR)(k−2) belonging to the directionscontained in the set

_(Ω,ACT) (k), and a residual ambient HOA component C_(AMB)(k−2). Thedelay of two frames is introduced as a result of overlap-add processingin order to obtain smooth signals. It is assumed that X_(DIR)(k−2) iscontaining a total of D channels, of which however only thosecorresponding to the active directional signals are non-zero. Theindices specifying these channels are assumed to be output in the dataset

_(DIR,ACT)(k−2) Additionally, the decomposition in step/stage 14provides some parameters ζ(k−2) which can be used at decompression sidefor predicting portions of the original HOA representation from thedirectional signals (see EP 13305156.5 for more details). In order toexplain the meaning of the spatial prediction parameters ζ(k−2), the HOAdecomposition is described in more detail in the below section HOAdecomposition.

In step or stage 15, the number of coefficients of the ambient HOAcomponent C_(AMB)(k−2) is reduced to contain onlyO_(RED)+D−N_(DIR,ACT)(k−2) non-zero HOA coefficient sequences, whereN_(DIR,ACT)(k−2)=|

_(DIR,ACT)(k−2)| indicates the cardinality of the data set

_(DIR,ACT)(k−2), i.e. the number of active directional signals in framek−2. Since the ambient HOA component is assumed to be always representedby a minimum number O_(RED) of HOA coefficient sequences, this problemcan be actually reduced to the selection of the remainingD−N_(DIR,ACT)(k−2) HOA coefficient sequences out of the possibleO−O_(RED) ones. In order to obtain a smooth reduced ambient HOArepresentation, this choice is accomplished such that, compared to thechoice taken at the previous frame k−3, as few changes as possible willoccur.

The final ambient HOA representation with the reduced number ofO_(RED)±N_(DIR,ACT)(k−2) non-zero coefficient sequences is denoted byC_(AMB,RED)(k−2). The indices of the chosen ambient HOA coefficientsequences are output in the data set

_(AMB,ACT)(k−2)

In step/stage 16, the active directional signals contained inX_(DIR)(k−2) and the HOA coefficient sequences contained inC_(AMB,RED)(k−2) are assigned to the frame Y̆(k−2) of I channels forindividual perceptual encoding as described in EP 13305558.2.

Perceptual coding step/stage 17 encodes the I channels of frame Y(k−2)and outputs an encoded frame Y̆(k−2).

According to the invention, following the decomposition of the originalHOA representation in step/stage 14, the spatial prediction parametersor side information data ζ(k−2) resulting from the decomposition of theHOA representation are losslessly coded in step or stage 19 in order toprovide a coded data representation ζ_(COD)(k−2), using the index set

_(DIR,ACT)(k) delayed by two frames in delay 18.

HOA Decompression

In FIG. 2 it is exemplary shown how to embed in step or stage 25 thedecoding of the received encoded side information data ζ_(COD)(k−2)related to spatial prediction into the HOA decompression processingdescribed in FIG. 3 of patent application EP 13305558.2. The decoding ofthe encoded side information data ζ_(COD)(k−2) is carried out beforeentering its decoded version ζ(k−2) into the composition of the HOArepresentation in step or stage 23, using the received index set

_(DIR,ACT)(k) delayed by two frames in delay 24.

In step or stage 21 a perceptual decoding of the I signals contained inY̆(k−2) is performed in order to obtain the I decoded signals in Ŷ(k−2).

In signal re-distributing step or stage 22, the perceptually decodedsignals in Ŷ(k−2) are re-distributed in order to recreate the frame{circumflex over (X)}_(DIR)(k−2) of directional signals and the frameĈ_(AMB,RED)(k−2) of the ambient HOA component. The information about howto re-distribute the signals is obtained by reproducing the assigningoperation performed for the HOA compression, using the index data sets

_(DIR,ACT)(k) and

_(AMB,ACT)(k−2)

In composition step or stage 23, a current frame Ĉ(k−3) of the desiredtotal HOA representation is re-composed (according to the processingdescribed in connection with FIG. 2b and FIG. 4 of PCT/EP2013/075559using the frame {circumflex over (X)}_(DIR)(k−2) of the directionalsignals, the set

_(DIR,ACT)(k) of the active directional signal indices together with theset

_(Ω,ACT)(k) of the corresponding directions, the parameters ζ(k−2) forpredicting portions of the HOA representation from the directionalsignals, and the frame Ĉ_(AMB,RED)(k−2) of HOA coefficient sequences ofthe reduced ambient HOA component.

Ĉ_(AMB,RED)(k−2) corresponds to component {circumflex over (D)}_(A)(k−2)in PCT/EP2013/075559, and

_(Ω,ACT)(k) and

_(DIR,ACT)(k) correspond to A_({circumflex over (Ω)})(k) inPCT/EP2013/075559, wherein active directional signal indices can beobtained by taking those indices of rows of A_({circumflex over (Ω)})(k)which contain valid elements. I.e., directional signals with respect touniformly distributed directions are predicted from the directionalsignals {circumflex over (X)}_(DIR)(k−2) using the received parametersζ(k−2) for such prediction, and thereafter the current decompressedframe Ĉ(k−3) is re-composed from the frame of directional signals{circumflex over (X)}_(DIR)(k−2), from

_(DIR,ACT)(k) and

_(Ω,ACT)(k), and from the predicted portions and the reduced ambient HOAcomponent Ĉ_(AMB,RED)(k−2).

HOA Decomposition

In connection with FIG. 3 the HOA decomposition processing is describedin detail in order to explain the meaning of the spatial predictiontherein. This processing is derived from the processing described inconnection with FIG. 3 of patent application PCT/EP2013/075559.

First, the smoothed dominant directional signals X_(DIR)(k−1) and theirHOA representation C_(DIR)(k−1) are computed in step or stage 31, usingthe long frame {tilde over (C)}(k) of the input HOA representation, theset

_(Ω,ACT)(k) of directions and the set

_(DIR,ACT)(k) of corresponding indices of directional signals. It isassumed that X_(DIR)(k−1) contains a total of D channels, of whichhowever only those corresponding to the active directional signals arenon-zero. The indices specifying these channels are assumed to be outputin the set

_(DIR,ACT)(k−1)

In step or stage 33 the residual between the original HOA representation{tilde over (C)}(k−1) and the HOA representation C_(DIR)(k−1) of thedominant directional signals is represented by a number of O directionalsignals {tilde over (X)}_(RES)(k−1), which can be considered as beinggeneral plane waves from uniformly distributed directions, which arereferred to a uniform grid.

In step or stage 34 these directional signals are predicted from thedominant directional signals X_(DIR)(k−1) in order to provide thepredicted signals {circumflex over ({tilde over (X)})}_(RES)(k−1)together with the respective prediction parameters ζ(k−1). For theprediction only the dominant directional signals x_(DIR,d)(k−1) withindices d, which are contained in the set

_(DIR,ACT)(k−1), are considered. The prediction is described in moredetail in the below section Spatial prediction.

In step or stage 35 the smoothed HOA representation Ĉ_(RES)(k−2) of thepredicted directional signals {circumflex over ({tilde over(X)})}_(RES)(k−1) is computed.

In step or stage 37 the residual C_(AMB)(k−2) between the original HOArepresentation {tilde over (C)}(k−2) and the HOA representationC_(DIR)(k−2) of the dominant directional signals together with the HOArepresentation Ĉ_(RES)(k−2) of the predicted directional signals fromuniformly distributed directions is computed and is output.

The required signal delays in the FIG. 3 processing are performed bycorresponding delays 381 to 387.

Spatial Prediction

The goal of the spatial prediction is to predict the O residual signals

$\begin{matrix}{{{\overset{\sim}{X}}_{RES}( {k - 1} )} = \begin{bmatrix}{{\overset{\sim}{x}}_{{RES},{GRID},1}( {k - 1} )} \\{{\overset{\sim}{x}}_{{RES},{GRID},2}( {k - 1} )} \\\vdots \\{{\overset{\sim}{x}}_{{RES},{GRID},O}( {k - 1} )}\end{bmatrix}} & (2)\end{matrix}$

from the extended frame

$\begin{matrix}\begin{matrix}{{{\overset{\sim}{X}}_{DIR}( {k - 1} )}:=\begin{bmatrix}{X_{DIR}( {k - 3} )} & {X_{DIR}( {k - 2} )} & {X_{DIR}( {k - 1} )}\end{bmatrix}} \\{= {\begin{bmatrix}{{\overset{\sim}{x}}_{{DIR},1}( {k - 1} )} \\{{\overset{\sim}{x}}_{{DIR},2}( {k - 1} )} \\\vdots \\{{\overset{\sim}{x}}_{{DIR},D}( {k - 1} )}\end{bmatrix}(4)}}\end{matrix} & (3)\end{matrix}$

of smoothed directional signals (see the description in above sectionHOA decomposition and in patent application PCT/EP2013/075559).

Each residual signal {tilde over (x)}_(RES,GRID,q)(k−1), q=1, . . . , O,represents a spatially dispersed general plane wave impinging from thedirection Ω_(q), whereby it is assumed that all the directions Ω_(q),q=1, . . . , O are nearly uniformly distributed over the unit sphere.The total of all directions is referred to as a ‘grid’.

Each directional signal {tilde over (x)}_(DIR,d)(k−1), d=1, . . . , Drepresents a general plane wave impinging from a trajectory interpolatedbetween the directions Ω_(ACT,d)(k−3), Ω_(ACT,d)(k−2), Ω_(ACT,d)(k−1)and Ω_(ACT,d)(k), assuming that the d-th directional signal is activefor the respective frames.

To illustrate the meaning of the spatial prediction by means of anexample, the decomposition of an HOA representation of order N=3 isconsidered, where the maximum number of directions to extract is equalto D=4. For simplicity it is further assumed that only the directionalsignals with indices ‘1’ and ‘4’ are active, while those with indices‘2’ and ‘3’ are non-active. Additionally, for simplicity it is assumedthat the directions of the dominant sound sources are constant for theconsidered frames, i.e.

Ω_(ACT,d)(k−3)=Ω_(ACT,d)(k−2)=Ω_(ACT,d)(k−1)−Ω_(ACT,d)(k)=Ω_(ACT,d) ford=1,4  (5)

As a consequence of order N=3, there are O=16 directions Ω_(q), ofspatially dispersed general plane waves {tilde over(x)}_(RES,GRID,q)(k−1), q=1, . . . , O. FIG. 4 shows these directionstogether with the directions Ω_(ACT,1) and Ω_(ACT,4) of the activedominant sound sources.

State-of-the-Art Parameters for Describing the Spatial Prediction

One way of describing the spatial prediction is presented in theabove-mentioned ISO/IEC document. In this document, the signals {tildeover (x)}_(RES,GRID,q)(k−1), q=1, . . . , O are assumed to be predictedby a weighted sum of a predefined maximum number D_(PRED) of directionalsignals, or by a low pass filtered version of the weighted sum. The sideinformation related to spatial prediction is described by the parameterset ζ(k−1)={p_(TYPE)(k−1), P_(IND)(k−1), P_(QF)(k−1)}, which consists ofthe following three components:

-   -   The vector p_(TYPE)(k−1) whose elements p_(TYPE,q)(k−1), q=1, .        . . , O indicate whether or not for the q-th direction Ω_(q) a        prediction is performed, and if so, then they also indicate        which kind of prediction. The meaning of the elements is as        follows:

$\begin{matrix}{{p_{{TYPE},q}( {k - 1} )} = ( {\begin{matrix}0 & {{for}\mspace{14mu} {no}\mspace{14mu} {prediction}\mspace{14mu} {for}\mspace{14mu} {direction}\mspace{14mu} \Omega_{q}} \\1 & {{for}\mspace{14mu} a\mspace{14mu} {full}\mspace{14mu} {band}\mspace{14mu} {prediction}\mspace{14mu} {for}\mspace{14mu} {direction}\mspace{14mu} \Omega_{q}} \\2 & {{for}\mspace{14mu} a\mspace{14mu} {low}\mspace{14mu} {band}\mspace{14mu} {prediction}\mspace{14mu} {for}\mspace{14mu} {direction}\mspace{14mu} \Omega_{q}}\end{matrix}.} } & (6)\end{matrix}$

-   -   The matrix P_(IND)(k−1), whose elements p_(IND,d,q)(k−1), d=1, .        . . , D_(PRED), q=1, . . . , O denote the indices from which        directional signals the prediction for the direction Ω_(q) has        to be performed. If no prediction is to be performed for a        direction Ω_(q), the corresponding column of the matrix        P_(IND)(k−1) consists of zeros. Further, if less than D_(PRED)        directional signals are used for the prediction for a direction        Ω_(q), the non-required elements in the q-th column of        P_(IND)(k−1) are also zero.    -   The matrix P_(QF)(k−1), which contains the corresponding        quantised prediction factors n p_(Q,F,d,q)(k−1), d=1, . . . ,        D_(PRED), q=1, . . . , O.

The following two parameters have to be known at decoding side forenabling the appropriate interpretation of these parameters:

-   -   The maximum number D_(PRED) of directional signals, from which a        general plane wave signal {tilde over (x)}_(RES,GRID,q)(k−1) is        allowed to be predicted.    -   The number B_(SC) of bits used for quantising the prediction        factors p_(Q,F,d,q)(k−1), d=1, . . . , D_(PRED), q=1, . . . , O.        The de-quantisation rule is given in equation (10).

These two parameters have to either be set to fixed values known to theencoder and decoder, or to be additionally transmitted, but distinctlyless frequently than the frame rate. The latter option may be used foradapting the two parameters to the HOA representation to be compressed.

An example for a parameter set may look like the following, assumingO=16, D_(PRED)=2 and B_(SC)=8

$\begin{matrix}{{{p_{TYPE}( {k - 1} )} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 2 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\end{bmatrix}},} & (7) \\{{{P_{IND}( {k - 1} )} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & 4 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\end{bmatrix}},} & (8) \\{{P_{QF}( {k - 1} )} = {\begin{bmatrix}40 & 0 & 0 & 0 & 0 & 0 & 15 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & {- 13} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\end{bmatrix}.}} & (9)\end{matrix}$

Such parameters would mean that the general plane wave signal {tildeover (x)}_(RES,GRID,1)(k−1) from direction Ω₁ is predicted from thedirectional signal {tilde over (x)}_(DIR,1)(k−1) from directionΩ_(ACT,1) by a pure multiplication (i.e. full band) with a factor thatresults from de-quantising the value 40. Further, the general plane wavesignal {tilde over (x)}_(RES,GRID,7)(k−1) from direction Ω₇ is predictedfrom the directional signals {tilde over (x)}_(DIR,1)(k−1) and {tildeover (x)}_(DIR,4)(k−1) by a lowpass filtering and multiplication withfactors that result from de-quantising the values 15 and −13.

Given this side information, the prediction is assumed to be performedas follows:

First, the quantised prediction factors p_(Q,F,d,q)(k−1),

d=1, . . . , D_(PRED), q=1, . . . , O are dequantised to provide theactual prediction factors

$\begin{matrix}{{p_{F,d,q}( {k - 1} )} = ( {\begin{matrix}{( {{p_{Q,F,d,q}( {k - 1} )} + \frac{1}{2}} )2^{{- B_{sc}} + 1}} & {{{if}\mspace{14mu} {p_{{IND},d,q}( {k - 1} )}} \neq 0} \\0 & {{{if}\mspace{14mu} {p_{{IND},d,q}( {k - 1} )}} = 0}\end{matrix}.} } & (10)\end{matrix}$

As already mentioned, B_(SC) denotes a predefined number of bits to beused for the quantisation of the prediction factors. Additionally,p_(F,d,q)(k−1) is assumed to be set to zero, if p_(IND,d,q)(k−1) isequal to zero.

For the previously mentioned example, assuming B_(SC)=8, thede-quantised prediction factor vector would result in

${P_{F}( {k - 1} )} \approx {\quad\begin{bmatrix}0.3164 & 0 & 0 & 0 & 0 & 0 & 0.1211 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & {- 0.0977} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\end{bmatrix}}$

Further, for performing a low pass prediction a predefined low pass FIRfilter

h _(LP) :=[h _(LP)(0)h _(LP)(1) . . . h _(LP)(L _(h)−1)]  (12)

of length L_(h)=31 is used. The filter delay is given by P_(h)=15samples.

Assuming as signals the predicted signals

$\begin{matrix}{{{\hat{\overset{\sim}{X}}}_{RES}( {k - 1} )} = \begin{bmatrix}{{\hat{\overset{\sim}{x}}}_{{RES},1}( {k - 1} )} \\{{\hat{\overset{\sim}{x}}}_{{RES},2}( {k - 1} )} \\\vdots \\{{\hat{\overset{\sim}{x}}}_{{RES},O}( {k - 1} )}\end{bmatrix}} & (13)\end{matrix}$

and the directional signals

$\begin{matrix}{{{\overset{\sim}{X}}_{DIR}( {k - 1} )} = \begin{bmatrix}{{\overset{\sim}{x}}_{{DIR},1}( {k - 1} )} \\{{\overset{\sim}{x}}_{{DIR},2}( {k - 1} )} \\\vdots \\{{\overset{\sim}{x}}_{{DIR},D}( {k - 1} )}\end{bmatrix}} & (14)\end{matrix}$

to be composed of their samples by

{circumflex over ({tilde over (x)})} _(RES,q)(k−1)=[{circumflex over({tilde over (x)})} _(RES,q)(k−1,1){circumflex over ({tilde over (x)})}_(RES,q)(k−1,2) . . . {circumflex over ({tilde over (x)})}_(RES,q)(k−1,2L)] for q=1, . . . ,O,  (15)

and

{tilde over (x)} _(DIR,d)(k−1)=[{tilde over (x)} _(DIR,d)(k−1,1){tildeover (x)} _(DIR,d)(k−1,2) . . . {tilde over (x)} _(DIR,d)(k−1,3L)] ford=1, . . . ,D,  (16)

the sample values of the predicted signals are given by

$\begin{matrix}{{{\hat{\overset{\sim}{x}}}_{{RES},q}( {{k - 1},l} )} = ( {\begin{matrix}0 & {{{if}\mspace{14mu} {p_{{TYPE},q}( {k - 1} )}} = 0} \\\begin{matrix}{\sum\limits_{d = 1}^{D_{PRED}}\; {{p_{F,d,q}( {k - 1} )} \cdot}} \\{{\overset{\sim}{x}}_{{DIR},{p_{{IN},d,q}{({k - 1})}}}( {{k - 1},{L + l}} )}\end{matrix} & {{{if}\mspace{14mu} {p_{{TYPE},q}( {k - 1} )}} = 1} \\{\sum\limits_{d = 1}^{D_{PRED}}\; {{p_{F,d,q}( {k - 1} )} \cdot {{\overset{\sim}{y}}_{{LP},q}( {{k - 1},l} )}}} & {{{if}\mspace{14mu} {p_{{TYPE},q}( {k - 1} )}} = 2}\end{matrix}{with}} } & (17) \\{\sum\limits_{j = 0}^{\min {({{L_{h} - 1},{l + {2\; D_{h}} - 1}})}}\; {{h_{LP}(j)} \cdot {{{\overset{\sim}{x}}_{{DIR},{p_{{IND},d,q}{({k - 1})}}}( {{k - 1},{L + l + D_{h} - j}} )}.}}} & (18)\end{matrix}$

As already mentioned and as now can be seen from equation (17), thesignals {tilde over (x)}_(RES,GRID,q)(k−1), q=1, . . . , O are assumedto be predicted by a weighted sum of a predefined maximum numberD_(PRED) of directional signals, or by a low pass filtered versions ofthe weighted sum.

State-of-the-Art Coding of the Side Information Related to SpatialPrediction

In the above-mentioned ISO/IEC document the coding of the spatialprediction side information is addressed. It is summarised in Algorithm1 depicted in FIG. 5 and will be explained in the following. For aclearer presentation the frame index k−1 is neglected in allexpressions.

First, a bit array ActivePred consisting of O bits is created, in whichthe bit ActivePred[q] indicates whether or not for the direction Ω_(q) aprediction is performed. The number of ‘ones’ in this array is denotedby NumActivePred.

Next, the bit array PredType of length NumActivePred is created whereeach bit indicates, for the directions where a prediction is to beperformed, the kind of the prediction, i.e. full band or low pass. Atthe same time, the unsigned integer array PredDirSigIds of lengthNumActivePred D_(PRED) is created, whose elements denote for each activeprediction the D_(PRED) indices of the directional signals to be used.If less than D_(PRED) directional signals are to be used for theprediction, the indices are assumed to be set to zero. Each element ofthe array PredDirSigIds is assumed to be represented by ┌log₂(D+1)┐bits. The number of non-zero elements in the array PredDirSigIds isdenoted by NumNonZeroIds.

Finally, the integer array QuantPredGains of length NumNonZeroIds iscreated, whose elements are assumed to represent the quantised scalingfactors P_(Q,F,d,q)(k−1) to be used in equation (17). The dequantisationto obtain the corresponding dequantised scaling factors P_(F,d,q)(k−1)is given in equation (10). Each element of the array QuantPredGains isassumed to be represented by B_(SC) bits.

In the end, the coded representation of the side information ζ_(COD)consists of the four aforementioned arrays according to

ζ_(COD)=[ActivePred PredType PredDirSigIds QuantPredGains].  (19)

For explaining this coding by an example, the coded representation ofequations (7) to (9) is used:

ActivePred=[1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0]  (20)

PredType=[0 1]  (21)

PredDirSigIds=[1 0 1 4]  (22)

QuantPredGains=[40 15 13].  (23)

The number of required bits is equal to 16+2+3 4+8 3=54.

Inventive Coding of the Side Information Related to Spatial Prediction

In order to increase the efficiency of the coding of the sideinformation related to spatial prediction, the state-of-the-artprocessing is advantageously modified.

-   -   A) When coding HOA representations of typical sound scenes, the        inventors have observed that there are often frames where in the        HOA compression processing the decision is taken to not perform        any spatial prediction at all. However, in such frames the bit        array ActivePred consists of zeros only, the number of which is        equal to O. Since such frame content occurs quite often, the        inventive processing prepends to the coded representation        ζ_(COD) a single bit PSPredictionActive, which indicates if any        prediction is to be performed or not. If the value of the bit        PSPredictionActive is zero (or ‘1’ as an alternative), the array        ActivePred and further data related to the prediction are not to        be included into the coded side information ζ_(COD)In practise,        this operation reduces over time the average bit rate for the        transmission of Z ζ_(COD).    -   B) A further observation made while coding HOA representations        of typical sound scenes is that the number NumActivePred of        active prediction is often very low. In such situation, instead        of using the bit array ActivePred for indicating for each        direction Ω_(q) whether or not the prediction is performed, it        can be more efficient to transmit or transfer instead the number        of active predictions and the respective indices. In particular,        this modified kind of coding the activity is more efficient in        case that

NumActivePred≤M _(M),  (24)

where M_(M) is the greatest integer number that satisfies

┌log₂(M _(M))┐+M _(M)·┌log₂(O)┐<0.  (25)

The value of M_(M) can be computed only with the knowledge of the HOAorder N:O=(N+1)² as mentioned above.

In equation (25), ┌log₂(M_(M))┐ denotes the number of bits required forcoding the actual number NumActivePred of active predictions, andM_(M)·┌log₂(O)┐ is the number of bits required for coding the respectivedirection indices. The right hand side of equation (25) corresponds tothe number of bits of the array ActivePred, which would be required forcoding the same information in the known way.

According to the aforementioned explanations, a single bitKindOfCodedPredIds can be used for indicating in which way the indicesof those directions, where a prediction is supposed to be performed, arecoded. If the bit KindOfCodedPredIds has the value ‘1’ (or ‘0’ in thealternative), the number NumActivePred and the array PredIds containingthe indices of directions, where a prediction is supposed to beperformed, are added to the coded side information ζ_(COD). Otherwise,if the bit KindOfCodedPredIds has the value ‘0’ (or ‘1’ in thealternative), the array ActivePred is used to code the same information.

On average, this operation reduces over time the bit rate for thetransmission of ζ_(COD).

C) To further increase the side information coding efficiency, the factis exploited that often the actually available number of activedirectional signals to be used for prediction is less than D. This meansthat for the coding of each element of the index array PredDirSigIdsless than ┌log₂(D+1)┐ bits are required. In particular, the actuallyavailable number of active directional signals to be used for predictionis given by the number {tilde over (D)}_(ACT) of elements of the dataset

_(DIR,ACT), which contains the indices {tilde over (ι)}_(ACT,1), . . . ,{tilde over (ι)}_(ACT,{tilde over (D)}) _(ACT) of the active directionalsignals. Hence, ┌log₂(|D_(ACT)+1|)┐ bits can be used for coding eachelement of the index array PredDirSigIds, which kind of coding is moreefficient. In the decoder the data set

_(DIR,ACT) is assumed to be known, and thus the decoder also knows howmany bits have to be read for decoding an index of a directional signal.Note that the frame indices of ζ_(COD) to be computed and the used indexdata set ζ_(DIR,ACT) have to be identical.

The above modifications A) to C) for the known side information codingprocessing result in the example coding processing depicted in FIG. 6.

Consequently, the coded side information consists of the followingcomponents:

$\begin{matrix}{\zeta_{COD} = ( \begin{matrix}\lbrack{PSPredictionActive}\rbrack & {{{if}\mspace{14mu} {PSPredictionActive}} = 0} \\\begin{bmatrix}{PSPredictionActive} \\{KindOfCodedPredIds} \\{ActivePred} \\{PredType} \\{PredDirSigIds} \\{QuantPredGains}\end{bmatrix} & \begin{matrix}{{{if}\mspace{14mu} {PSPredictionActive}} = {1}} \\{{KindOfCodedPredIds} = 0}\end{matrix} \\\begin{bmatrix}{PSPredictionActive} \\{KindOfCodedPredIds} \\{NumActivePred} \\{PredIds} \\{PredType} \\{PredDirSigIds} \\{QuantPredGains}\end{bmatrix} & \begin{matrix}{{{if}\mspace{14mu} {PSPredictionActive}} = {1}} \\{{KindOfCodedPredIds} = 1}\end{matrix}\end{matrix} } & (26)\end{matrix}$

Remark: in the above-mentioned ISO/IEC document e.g. in section 6.1.3,QuantPredGains is called PredGains, which however contains quantisedvalues.

The coded representation for the example in equations (7) to (9) wouldbe:

PSPredictionActive=1  (27)

KindOfCodedPredIds=1  (28)

NumActivePred=2  (29)

PredIds=[1 7]  (30)

PredType=[0 1]  (31)

PredDirSigIds=[1 0 1 4]  (32)

QuantPredGains=[40 15 −13],  (33)

and the required number of bits is 1+1+2+2 4+2+2 4+8 3=46.

Advantageously, compared to the state of the art coded representation inequations (20) to (23), this representation coded according to theinvention requires 8 bits less.

Decoding of the Modified Side Information Coding Related to SpatialPrediction

The decoding of the modified side information related to spatialprediction is summarised in the example decoding processing depicted inFIG. 7 and FIG. 8 (the processing depicted in FIG. 8 is the continuationof the processing depicted in FIG. 7) and is explained in the following.

Initially, all elements of vector p_(TYPE) and matrices P_(IND) andP_(Q,F) are initialised by zero. Then the bit PSPredictionActive isread, which indicates if a spatial prediction is to be performed at all.In the case of a spatial prediction (i.e. PSPredictionActive=1), the bitKindOfCodedPredIds is read, which indicates the kind of coding of theindices of directions for which a prediction is to be performed.

In the case that KindOfCodedPredIds=0, the bit array ActivePred oflength O is read, of which the q-th element indicates if for thedirection Ω_(q) a prediction is performed or not. In a next step, fromthe array ActivePred the number NumActivePred of predictions is computedand the bit array PredType of length NumActivePred is read, of which theelements indicate the kind of prediction to be performed for each of therelevant directions. With the information contained in ActivePred andPredType, the elements of the vector p_(TYPE) are computed.

In case KindOfCodedPredIds=1, the number NumActivePred of activepredictions is read, which is assumed to be coded with ┌log₂(M_(M))┐bits, where M_(M) is the greatest integer number satisfying equation(25). Then, the data array PredIds consisting of NumActivePred elementsis read, where each element is assumed to be coded by ┌log₂(O)┌ bits.The elements of this array are the indices of directions, where aprediction has to be performed. Successively, the bit array PredType oflength NumActivePred is read, of which the elements indicate the kind ofprediction to be performed for each one of the relevant directions. Withthe knowledge of NumActivePred, PredIds and PredType, the elements ofthe vector p_(TYPE) are computed.

For both cases (i.e. KindOfCodedPredIds=0 and KindOfCodedPredIds=1), inthe next step the array PredDirSigIds is read, which consists ofNumActivePred·D_(PRED) elements. Each element is assumed to be coded by┌log₂({tilde over (D)}_(ACT))┐ bits. Using the information contained inp_(TYPE),

_(DIR,ACT) and PredDirSigIds, the elements of matrix R_(IND) are set andthe number NumNonZeroIds of non-zero elements in R_(IND) is computed.

Finally, the array QuantPredGains is read, which consists ofNumNonZeroIds elements, each coded by B_(SC) bits. Using the informationcontained in R_(IND) and QuantPredGains, the elements of the matrixP_(QF) are set.

The inventive processing can be carried out by a single processor orelectronic circuit, or by several processors or electronic circuitsoperating in parallel and/or operating on different parts of theinventive processing.

1. A method for decoding a bitstream including encoded HOArepresentations, said method comprising: evaluating a value of a bitKindOfCodedPredIds; evaluating, based on the value of the bitKindOfCodedPredIds, a first array ActivePred, wherein each element ofthe first array ActivePred indicates if, for a corresponding direction,a prediction is performed; determining, based on the evaluation of thefirst array ActivePred, elements of a vector p_(type); evaluating asecond array PredDirSigIds, wherein elements of the second arrayPredDirSigIds denote indices of directional signals to be used foractive predictions; determining, based on the vector p_(type) and theelements of the second array PredDirSigIds, elements of a matrix PINEdenoting indices from which directional signals a prediction for adirection is to be performed.
 2. A method according to claim 1, whereineach element of the second array PredDirSigIds denotes, for thepredictions to be performed, indices of the directional signals to beused and wherein each element was coded based on ┌log₂(|{tilde over(D)}_(ACT)+1|)┐ bits, and is correspondingly decoded, wherein {tildeover (D)}_(ACT) denotes a number of elements of said data set of indicesof directional signals.
 3. An apparatus decoder for decoding a bitstreamincluding encoded HOA representations, said method apparatus comprising:a processor configured to: evaluate a value of a bit KindOfCodedPredIds;evaluate, based on the value of the bit KindOfCodedPredIds, a firstarray ActivePred, wherein each element of the first array ActivePredindicates if, for a corresponding direction, a prediction is performed;determine, based on the evaluation of the first array ActivePred,elements of a vector p_(type); evaluate a second array PredDirSigIds,wherein elements of the second array PredDirSigIds denote indices ofdirectional signals to be used for active predictions; determine, basedon the vector p_(type) and the elements of the second arrayPredDirSigIds, elements of a matrix PIND denoting indices from whichdirectional signals a prediction for a direction is to be performed. 4.A non-transitory computer readable storage medium containinginstructions that when executed by a processor perform a method ofdecoding a bitstream including encoded HOA representations, said methodcomprising: evaluating a value of a bit KindOfCodedPredIds; evaluating,based on the value of the bit KindOfCodedPredIds, a first arrayActivePred, wherein each element of the first array ActivePred indicatesif, for a corresponding direction, a prediction is performed;determining, based on the evaluation of the first array ActivePred,elements of a vector p_(type); evaluating a second array PredDirSigIds,wherein elements of the second array PredDirSigIds denote indices ofdirectional signals to be used for active predictions; determining,based on the vector p_(type) and the elements of the second arrayPredDirSigIds, elements of a matrix P_(IND) denoting indices from whichdirectional signals a prediction for a direction is to be performed.